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PURPOSE 


In  May  1992,  the  Secretary  of  Defense  forwarded  the  Department  of  Defense  (DoD) 
High  Performance  Computing  (HPC)  Modernization  Plan  to  Congress.  That  plan, 
prepared  by  the  Office  of  the  Director  of  Defense  Research  and  Engineering  (DDR&E)  and 
published  in  March  1992,  provided  the  rationale,  process,  timetable,  and  funding 
requirements  for  high  peiformance  computing  modernization  supporting  E>oD  research, 
development,  test,  and  evaluation  (RDT&E).  The  purpose  of  this  document  is  to  update 
and  revise  the  Modernization  Plan  to  reflect  the  activities  of  the  past  two  years,  and  to 
present  FY94,  FY95,  and  outyear  plans. 


The  DoD  HPC  Modernization  program  is  structured  to  modernize  the  total  high 
performance  computational  capability  of  DoD  research  and  development  (R&D)  to  a  level 
comparable  to  that  available  in  the  foremost  civilian  and  other  Government  agency  R&D 
environments.  The  program  is  multifaceted — creating  robustly  networked  IffC 
environments  at  Major  Shared  Resource  Centers,  accommodating  specialized  HPC 
requirements  at  Distributed  Centers,  adapting  and  developing  scalable  applications 
software,  and  educating  users  as  new  architectures  and  concepts  evolve.  The  program 
fosters  meaningful  interaction  with  the  entire  HPC  community  to  facilitate  sharing 
knowledge,  tools,  and  expertise. 


SCOPE 

This  plan  covers  HPC  in  support  of  the  DoD  science  and  technology  program  under 
the  cognizance  of  the  DDR&E.  It  dso  stimulates  interaction  and  coordination  with  other 
DoD  and  external  R&D  activities  and  promotes  the  incorporation  of  HPC  technology  into 
DoD  operational  environments. 


Executive  Summary 


The  Department  of  Defense  relies  on  technologically  superior  systems  to  counter 
numerically  larger  forces,  to  reduce  casualties  and  damage  to  the  national  infra-structure, 
and  to  permit  rapid,  decisive  action  when  necessary.  Wlule  DoD  is  facing  significant 
reductions  in  force  structure,  the  need  to  maintain  technical  superiority  is  more  critical  than 
ever.  Therefore  it  follows  that  it  is  necessary  to  sustain  the  investment  in  Science  and 
Technology  (S&T)  and  the  supporting  key  technology  program — the  DoD  High 
Performance  Computing  Modernization  Program. 

The  success  of  the  HPC  Modernization  Program  will  critically  impact  the  conduct 
of  the  Department's  Science  and  Technology  initiatives  as  it  addresses  the  crucial  next  step 
of  integrating  and  deploying  advances  in  computing  and  communications  into  DoD  S&T 
programs,  and  couples  them  with  military  systems  design  and  development. 

The  DoD  HPC  Modernization  Program  is  structured  to  modernize  the  total  high 
performance  computational  capability  of  DoD  research  and  development  (R&D)  to  a  level 
comparable  to  that  available  in  the  foremost  civilian  and  other  Government  agency  R&D 
environments.  This  is  in  consonance  with  the  Congressional  direction  to  DoD  to  program 
and  budget  for  HPC  modernization  of  its  laboratories  in  recognition  that  the  HPC  resources 
and  environments  available  to  defense  scientists  and  engineers  had  fallen  below  the  levels 
available  to  their  counterparts  in  the  civilian  sector. 

The  resultant  DoD  HPC  Modernization  Program  is  a  coordinated,  sustained 
initiative  to  develop  complete,  balanced  HPC  environments  based  on  user  requirements. 
The  planned  HPC  capability  is  focused  upon  the  establishment  of  networked  Major  Shared 
Resource  Centers  (MSRCs)  and  Distributed  Centers.  The  MSRCs  provide  complete  HPC 
environments  including  various  types  of  computing  systems,  scientific  visualization 
capabilities,  extensive  peripheral  and  archival  storage,  and  expertise  in  use  of  these 
systems.  The  MSRCs  support  the  wide  variety  of  research  and  development  problems 
arising  from  the  S&T  program.  The  MSRCs  address  the  computer  and  computational 
sciences,  and  provide  Ae  education  and  training  which  allows  all  of  the  DoD  laboratories  to 
be  more  productive  in  S&T  research  and  development.  In  addition  to  the  MSRCs,  the 
program  will  deploy  systems  at  Distributed  Centers  to  fulfill  special  R&D  requirements 
where  deployments  of  the  HPC  systems  are  critical  to  success  of  R&D  efforts. 

Initiatives  and  mechanisms  are  being  established  to  ensure  that  the  modernization  of 
the  DoD  laboratories  is  interrelated,  well  formulated,  and  within  the  context  of  the  national 
HPC  infrastructure.  The  program  will  take  actions  to  encourage  and  facilitate  interactions, 
collaborations,  and  communications  with  the  civilian  HPC  community  and  other 
government  agencies  to  share  knowledge,  tools,  and  expertise. 

The  program  is  built  upon  the  foundations  of  proper  planning  and  management 
The  program  vision,  goals,  strategies,  and  metrics  are  described  herein  in  detail,  and,  with 
the  r^uirements  analysis  (summary  herein),  form  the  basis  for  modernization  decisions 
and  acquisition  plans/strategies.  A  Joint  High  Performance  Computing  Modernization 
Office  was  formed  to  function  in  an  intellectual  and  leadership  role  for  defining  the 
direction,  planning,  and  policy-making  for  DoD  HPC  Modernization.  T^is  office  will 
perform  the  necessary  life  cycle  management  functions  and  the  acquisition  streamlining 
processes,  as  required,  to  implement  and  execute  the  program  in  an  effective  and  cost 
efficient  manner. 

Finally,  the  plan  identifies  the  current  and  following  five  year  funding  requirements 
to  enable  the  ongoing  DoD  R&D  activities  to  meet  mission-driven  requirements  and  to 
fully  engage  the  national  HPC  computational  and  networking  trends. 
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1 .  Introduction 


In  March  1992,  the  Office  of  the  Director.  Defense  Research  and  Engineering 
(DDR&E)  published  the  Department  of  Defense  (DoD)  High  Performance  Computing 
(HPC)  Modernization  Plan.  The  purpose  of  the  plan  was  to  provide  the  rationale,  process, 
timetable,  and  funding  requirements  for  high  performance  computing  modernization  in 
DoD  research,  development,  test,  and  engineering  under  the  cognizance  of  the  DDR&E. 
This  document  updates  the  DoD  HPC  Modernization  Plan,  reflecting  the  activities  of  the 
program  to  date,  and  the  plans  necessary  to  meet  the  Congressional  modernization 
mandate.  The  document  is  structured  to  provide  the  reader  with  a  review  of  DoD  mission 
needs,  the  program  genesis,  and  the  recent  program  realignment.  The  plan  then  addresses 
the  program  foundations:  vision,  goals,  strategies,  and  metrics.  A  summary  of  a  recently 
completed  comprehensive  DoD  FffC  r^uirements  analysis  is  provided,  foUowed  by  the 
FY94,  FY95,  and  outyear  implementation  plans  to  satisfy  these  requirements.  Finally,  the 
funding  required  to  address  mission-driven  requirements  is  presented. 


1.1  Mission  Needs 

The  mission  of  this  program  is  to  modernize  the  high  performance  computational 
capability  of  DoD  laboratories  to  a  level  comparable  to  that  available  in  the  foremost  civilian 
and  other  Government  agency  R&D  environments.  The  program  is  multifaceted.  It 
involves  creating  HPC  environments  at  major  and  distributed  centers  accessible  to  local  and 
remote  scientists  and  engineers  via  high-speed  networks.  Providing  for  the  adaptation  of 
existing  and  development  of  new  scalable  applications  and  algorithms  to  address  growing 
science  and  technology  requirements,  along  with  continued  education  of  users  as  new 
architectures  and  concepts  evolve,  are  integral  components  of  the  program.  The  program 
pursues  continuous  interaction  with  the  national  HTC  infrastructure,  including  academe, 
indust^,  and  other  government  agencies  to  facilitate  the  sharing  of  knowledge,  tools,  and 
expertise. 

High  performance  computing  has  been  identified  as  a  key  enabling  technology 
essential  to  achieving  the  objectives  of  the  Department’s  science  and  technology  (S&TO 
program.  DoD  intends  to  exploit  the  orders  of  magnitude  improvements  in  computational 
and  communications  capabilities  as  a  result  of  advancement  in  hardware,  architectural 
designs,  software,  networking,  and  computational  methods  in  support  of  DoD  missions. 

Many  Defense-related  problems  in  the  areas  of  chemistry  and  materials  science, 
computational  fluid  dynamics  of  air,  undersea,  and  surface  vessels  and  their  ordnance, 
parametric  weight/vulnerability  reduction  studies,  automatic  target  recognition,  submarine 
hydrodynamics,  ocean  acoustics,  high  performance  weapons  design,  and  prediction  and 
dispersion  of  hazardous  materials,  require  computational  performance  of  trillions  of 
operations  per  second  (teraops).  These  diverse  S&T  applications  also  require  massive  data 
storage  and  scientific  visualization  capabilities  to  render  the  computation^  results  in 
graphical  images  that  enable  understanding  of  the  computations.  Hie  reliance  of  S&T 
research  on  computational  science  is  increasing  rapidly  due  to  the  greatly  expanding  use  of 
HPC  in  scientific  research,  modeling,  and  simulation.  HPC  greatly  benefits  DoD 
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laboratories  by  decreasing  the  need  for  prototype  systems  and  limiting  the  amount  of  live- 
fire  tests.  Solutions  to  critical  problems  leading  to  advancements  in  these  areas  are 
fundamentally  driven  or  limited  by  the  HPC  capability  and  environment  within  the  Defense 
laboratories.  DoD  S&T  researchers  require  Hrc  environments  populated  with  a  variety  of 
system  architectures  and  data  storage  to  permit  the  highest  fidelity  and  greatest  speed  in 
analyzing  problems  of  ever  increasing  size  and  complexity. 

For  these  reasons,  it  is  critical  that  DoD  vigorously  continue  implementation  of  this 
modernization  plan.  While  some  recent  improvement  has  been  achiev^  by  the  program, 
HPC  systems  currently  available  to  most  DoD  users  significantly  lag  behind  those 
commercially  available.  Current  data  storage  and  scientific  visudization  capabilities  are 
undersized  and  inadequate  for  today's  research  problems.  Many  science  problems, 
modeling  and  simulation  applications  require  HPC  systems  capable  of  scaling  to  several 
orders  of  magnitude  in  performance  beyond  those  being  used  by  DoD  laboratories  today. 
Software  initiatives  are  needed  in  applications  and  algorithm  development  to  enable  use  of 
the  latest  scalable  technologies.  Placement  of  these  new  technologies  in  the  laboratories 
requires  renewed  emphasis  on  new  software  techniques,  computational  and  computer 
science,  and  on  education  and  training.  All  of  these  areas  are  fundamental  to  effective  HPC 
capability  and  must  be  modernized  and  upgraded  in  parallel  to  meet  even  today’s 
requirements  of  the  DoD  S&T  community. 

DoD  will  need  to  provide  seamless  HPC  capabilities  to  scientists  and  engineers 
(S&E)  throughout  DoD  regardless  of  location  of  resources.  To  support  the  requirements  of 
remote  users,  an  advanced  high-speed,  robustly  interconnected  network  linking  DoD  S&T 
researchers  to  the  advanced  HPC  systems  is  needed.  This  network  capability  will  require 
continued  advancement  to  remain  consistent  with  the  performance  of  the  computing 
systems,  the  growing  data  traffic  requirements,  and  rapidly  evolving  network  technology. 

Finally,  to  fully  achieve  the  Department’s  scientific  computational  needs, 
applications,  software  tools,  and  algorithm  development  must  be  advanced  and  expanded  to 
lice  advantage  of  the  emerging  computing  power.  The  DoD  HPC  Modernization  Program 
must  stimulate  the  efforts  of  its  own  researchers  and  engage  and  leverage  the  efforts  of 
academe  and  industry  to  improve  the  ability  to  effectively  address  HPC  software  problems. 


1.2  Background 

In  1991  Congress  noted  that  DoD  laboratories  lagged  considerably  behind  the 
Department  of  Energy  (DOE),  National  Aeronautical  and  Space  Agency  (NASA),  and 
National  Science  Foundation  (NSF)  Supercomputer  Centers  in  computing  capability.  This 
prompted  the  enactment  of  language  in  the  National  Defense  Authorization  Act  of  FY92-93 
directing  DoD  to  present  a  five  year  master  plan  to  Congress  to  modernize  of  the  HPC 
capabilities  of  the  DoD  laboratories.  The  DDR&E  established  the  HPC  Working  Group 
(HPCWG)  to  develop  this  plan.  The  HPCWG  consisted  of  representatives  from  the 
Services,  Defense  Nuclear  Agency  (DNA),  Advanced  Research  Projects  Agency  (ARP A), 
and  the  Office  of  the  Assistant  Secretary  of  Defense  for  Command,  Control, 
Communications,  and  Intelligence  (ASD(C3I)).  The  Secretary  of  Defense  forwarded  the 
original  DoD  HPC  Modernization  Plan  to  Congress  in  May  1^2.  A  FY93  Implementation 
Plan  and  the  HPC  Modernization  Program  were  initiated  using  funds  that  Congress  had 
authorized  and  appropriated  (FY92  (M2M)  and  FY93  ($45M))  for  DoD  laboratory  HPC 
modernization. 

During  FY93  a  large  general  purpose  vector  supercomputer,  a  Cray  C9 16/5 12,  was 
acquired  and  installed  at  the  Army  Corps  of  Engineers  Waterways  Experiment  Station 
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(CEWES),  Vicksburg,  MS.  In  addition,  the  following  six  scalable  parallel  systems  were 
installed  or  upgraded: 


•  Intel  Paragon  XP/S- 1 5.  Aeronautical  Systems  Center  { ASC),  Wright-Patterson 
AFB,  Ohio 

•  Intel  Paragon  XP/S-2S,  Naval  Command,  Control  and  Ocean  Surveillance 
Center  (NCCOSC),  San  Diego,  California 

•  IBM  SPl,  Maui  HPC  Center  (MHPCC)  (supporting  the  Air  Force  Maui 
Optical  Station  (AMOS)),  Maui,  Hawaii 

•  Kendall  Square  Research  KSR 1-256,  Army  Research  Laboratory  (ARL), 
Aberdeen  Proving  Ground,  Maryland 

•  TMC  CM-5  enhanced.  Naval  Research  Laboratory  (NRL).  Washington,  DC 

•  TMC  CM-5  enhanced.  Army  High  Performance  Computing  Research  Clenter, 
(AHPCRC)  University  of  Minnesota,  Minneapolis,  Minnesota 

In  the  area  of  connectivity,  a  high  speed  wide  area  network  was  established  by 
integrating  and  supplementing  existing  Army  and  Air  Force  supercomputer  networks  to 
create  the  Interim  Defense  Research  and  Engineering  Network  (IDREN).  'fhis  network, 
illustrated  in  Figure  1,  interconnects  defense  S&T  users  and  DoD  HPC  resources  and 
provides  gateway  connectivity  to  the  Internet 


Figure  1.  interim  DREN  Configuration  -  3Q  FY94 
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Finally,  during  FY93.  various  upgrades  were  made  to  computing  systems  at  the 
National  Consortium  for  High  Performance  Computing  (NCHPC)  facilities.  In  return  DoD 
was  given  access  to  NCHPC’s  computing  resources  and  a  Memorandum  of  Agreement 
(MOA)  was  put  in  place  supporting  DoD's  HPC  needs  in  applications  software 
development  The  agreement  defines  six  DoD  technology  thrust  areas: 

'  Computational  Structural  Mechanics 

•  Computational  Fluid  Mechanics 

•  Computational  Chemistry  and  Materials  Science 

•  Computational  Electromagnetics  and  Acoustics 

•  Climate  /  Weather  /  Ocean  Modeling 

•  Signal  /  Image  Processing 

NCHPC  will  collaborate  with  the  Modernization  Program  to  accelerate  the 
exploitation  of  scalable  parallel  systems  to  solve  HPC  problems  in  these  technology  areas. 
TIk  MOA  provides  the  DoD  S&T  community  a  training  and  information  distribution 
infrastructure  available  through  NCHPC  and  access  for  DoD  researchers  to  NCHPC 
computers.  NCHPC’s  MOSAIC  information  system  will  be  used  by  DoD  to  access  these 
NCHPC  training  and  HPC  resources,  access  to  NCHPC  HPC  will  facilitate  DoD 
researcher  collaboration  with  NCHPC  academics  and  will  allow  scaling  up  to  much  larger 
scalable  HPC  in  some  cases. 


1.3  Program  Formalization 

In  recognition  of  the  program’s  potential  impact,  magnitude  of  the  effort,  and  the 
need  for  a  more  formal  management  structure,  the  DDR&E  appointed  an  Executive  Director 
and  a  Program  Manager  for  the  program  in  January,  1994.  A  HPC  Modernization 
(HPCM)  Office  was  created  and  staffed  with  representatives  from  each  Service  to  review 
the  program’s  vision  and  strategies  to  ensure  that  the  program  meets  the  laboratories'  and 
R&D  centers’  long-term  needs.  The  resultant  program  addresses  the  total  HPC 
environment  including  both  hardware  and  software  issues.  The  HPCM  office  will 
continue  to  function  in  an  intellectual  and  leadership  role  by  defining  the  direction, 
planning,  and  policy  for  DoD  HPC  modernization.  This  office  also  performs  the  necessary 
life  cycle  management  functions  as  required  to  implement  and  execute  the  program  in  an 
effective  and  cost  efficient  manner. 

The  high  level  management  structure  for  the  HPC  Modernization  Program  is 
illustrated  in  Figure  2. 
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Figure  2.  HPC  Modernization  Program  Management  Structure 


2 .  Program  Foundations 


DoD  HPC  modernization  is  fundamentally  driven  by  Defense  S&T  community 
requirements.  This  program  will  empower  the  DoD  researcher  to  meet  the  computational 
challenges  of  the  research  and  development  (R&D)  mission.  To  provide  an  integrated  cost 
efficient  solution,  the  modernization  has  been  planned  within  the  context  of  a  set  of 
guiding  principles  and  precepts  which  can  be  termed  vision,  goals,  program  strategy,  and 
metrics. 


2.1  Vision 

The  HPC  modernization  vision  is  to  raise  DoD' s  R&D  high  performance  computing 
and  communications  capacity  and  capabilities  to  a  level  equal  to  or  greater  than  that 
available  in  the  foremost  academic  research  centers  and  industry,  thereby  enabling  the 
United  States  to  maintain  its  technological  supremacy  in  weapon  systems  design  and  to 
foster  the  flow  of  this  technology  into  warfighting  support  systems  and  dual-use  functions. 

Successfully  fulfilling  this  vision  means  every  DoD  S&T  researcher  will  have  ready 
desktop  access  to  the  HPC  tools  necessary  to  complete  the  mission  in  a  timely  fashion. 

This  implies  that  DoD  researchers  have  access  via  their  desktop  systems  to  a  broad  HPC 
infrastructure  which  provides: 

•  Robust  high-speed  networking 

•  Remote  classified  and  unclassified  access  through  transparent  network  services 

•  A  variety  of  powerful  and  easy  to  use  systems  and  applications  software  to 
permit  optimum  mapping  of  system  types  to  particular  applications 

•  Massive,  hierarchical  data  storage 

•  Immediate  access  to  data 

•  Post-processing  to  facilitate  meaningful  data  interpretation 


2.2  Goals 

The  DoD  HPC  modernization  program  will  strategically  locate,  rapidly  deploy,  and 
sustain  the  highest  quality  high  performance  computing  environments  and  networks.  The 
emphasis  is  on  leveraging  the  national  trends  fostered  by  existing  DoD  investments  in  the 
Federal  High  Performance  Computing  and  Communications  (HPCC)  Program,  Defense 
HPC  R&D,  and  other  related  research,  development,  test,  and  evaluation  (RDT&E)  efforts. 
It  is  a  sustained  program  to  fully  engage  and  draw  from  the  national  effort 


Program  initiatives  are  directed  toward  the  following  goals: 

•  Establish  and  improve  the  access  to  state  of  the  art  HPC  capacity  and  capability 
for  DoD  scientists,  engineers,  and  analysts  to  enable  weapons  development 
and  more  capable  warfighting  systems. 

•  Encourage  development  of  software  tools  and  applications  to  make  it  easier  to 
use  and  exploit  the  capabilities  of  HPC,  and  expand  and  educate  the  DoD  HPC 
user  base. 

•  Engage,  leverage,  contribute  to  and  be  a  major  participant  in  the  National  HPC 
Infrastructure  and  exploit  its  benefits  for  Defense  R&D.  This  engagement 
includes  DoD  laboratories,  the  Services'  in-house  basic  research  entities  (Army 
Research  Office,  Air  Force  Office  of  Scientific  Research,  and  the  Office  of 
Naval  Research),  the  Advanced  Research  Projects  Agency,  industry, 
universities,  and  other  Government  agencies  such  as  the  Department  of 
Energy,  the  National  Science  Foundation,  and  the  National  Aeronautics  and 
Space  Administration. 

2.3  Strategies 

True  modernization  of  DoD's  HPC  capability  and  fulfillment  of  the  above  vision 
and  goals  requires  a  program  strategy  that  addresses  all  aspects  of  HPC.  While  advancing 
the  level  of  hardware  performance  is  critical  to  success,  the  higher  objective  is  to  enable 
better  scientific  research  and  technology  development  for  superior  weapons,  warfighting 
and  related  support  systems. 

The  HPC  assets  acquired  via  this  program  will  be  available  to  scientists  and 
engineers  supporting  the  DoD  S&T  program  in  the  I  ,fvices  and  Defense  agencies,  as  well 
as  collaborative  researchers  in  academe  and  U.S.  industry  supporting  the  S&T  program. 
These  HPC  resources  will  be  interconnected  via  the  networking  initiative  being 
implemented  under  the  DoD  HPC  Modernization  Program,  the  Defense  Research  and 
Engineering  Network  (DREN).  DREN  will  be  interconnected  to  the  Internet  to  allow  DoD 
scientists  and  engineers  to  properly  and  adequately  interface  to  their  counterparts  and 
colleagues  in  the  civilian  sector. 

The  strategy  to  improve  availability  and  access  to  state  of  the  art  HPC  capacity  and 
capability  encompasses: 

•  Providing  a  balanced  set  of  HPC  commercially  available  technologies  to  meet 
the  full  range  of  DoD  requirements  and  to  permit  optimum  mapping  of 
requirements  to  system  types. 

•  Building  complete  HPC  environments  at  DoD  laboratories  and  R&D  centers 
designated  as  HPC  Major  Shared  Resource  Centers  (MSRCs)  to  support  the 
widest  community  of  users.  These  MSRCs  will  include  hardware,  data 
storage,  scientific  visualization  capabilities,  high  speed  interconnections, 
supporting  infrastructure,  and  expertise  in  computational  and  computer 
sciences  and  HPC  software. 

•  .  Supporting  development  efforts  designed  to  provide  teracomputing  capabilities 
and  technologies  which  are  applicable  to  Defense  S&T  requirements. 


•  Developing  heterogeneous  computing  environments  to  allow  the  optimum 
mapping  of  varied  S&T  problems  onto  appropriate  types  of  HPC  systems. 

•  Provisioning,  where  appropriate,  special  configurations,  designs,  or 
architectures;  embedded  systems;  classified  systems;  distributed  data  storage; 
or  scientific  visualization  platforms  at  laboratories  designated  as  Distributed 
Centers.  These  Distributed  Centers  will  focus  on  and  primarily  address 
smaller  scale,  intense,  local  mission  needs  and  also  make  unus^  capacity  and 
capability  available  to  remote  users.  In  addition.  Distributed  Centers  will  allow 
leveraging  of  special  collections  of  HPC  expertise  for  the  benefit  of  the  entire 
program. 

•  Providing  a  focus  on  standards  in  DoD  S&T  software  to  ensure  that  future 
transitions  and  advancements  in  software  technology  are  applied  in  an  efficient 
and  cost  effective  manner. 

The  strategy  to  encourage  the  development  of  software  tools  and  applications  and 
expand  and  educate  the  DoD  HPC  user  base  includes: 

•  Focusing  on  applications  oriented  software  initiatives  designed  to  overcome  the 
technological  inhibitors  which  delay  the  effective  use  of  many  scalable  parallel 
high  performance  computers. 

•  Providing  user  education  and  training  in  computer  and  computational  sciences, 
software  applications  and  optimization,  and  code  conversion. 

•  Promoting  the  formation  of  DoD-sponsored  inter-disciplinary  teams  and 
collaboration  groups  to  leverage  academe  and  industry  expertise  in  HPC  for  the 
solution  of  DoD  problems. 

•  Deploying  robust  and  cost  effective  network  connectivity  consistent  with  HPC 
traffic  requirements  among  and  between  DoD  researchers  and  HPC  assets,  and 
between  ^e  DoD  and  external  R&D  communities. 

The  strategy  to  engage,  leverage,  contribute  to,  and  be  a  major  participant  in  the 
national  HPC  infrastructure  includes: 

•  Ensuring  the  Defense  laboratories  remain  cognizant  of,  interact  with,  and 
leverage  internal  DoD  HPC  initiatives  such  as  the  ARPA  technology 
development  efforts  and  the  algorithm  and  software  research  initiatives  of  the 
Defense  research  offices. 

•  Ensuring  that  the  Defense  laboratories,  remain  cognizant  of,  collaborate  with, 
and  leverage  other  government  agencies’  and  each  other’s  HPC  efforts. 

•  Providing  HPC  environments  to  support  defense  applications  which  are 
directly  related  to  dual  use  technologies  and  National  Challenges,  e.g., 
environment,  medical,  automotive,  and  manufacturing  processes  and  products. 

•  Ensuring  the  availability  of  HPC  environments  to  support  Defense 
applications  which  address  Grand  Challenge  needs,  e.g.,  meteorological, 
oceanographies,  and  environmental  modeling,  nature  of  new  materids, 
structure  of  biological  and  chemical  phenomena,  and  energy  efficient  vehicles 
and  airplanes. 


2.4  Metrics 


The  metrics  for  this  program  are  closely  tied  to  the  program  goals.  The  ultimate 
measure  of  success  will  be  improved  quality  of  research  within  the  S&T  community,  e.g. 
more  realistic  models  and  quicker  solutions  to  DoD  S&T  problems.  DoD  R&D  HPC 
capability  and  capacity  will  also  be  measured  and  compared  relative  to  that  available  to  the 
non-Defense  HPC  community.  The  success  of  the  program  in  meeting  this  goal  will  be 
demonstrated  by  peer  group  evaluations. 

In  addition,  the  level  to  which  DoD  HPC  capabilities  satisfy  the  computational 
requirements  of  the  S&T  program  will  be  monitored  not  only  for  computational  speed,  but 
also  for  all  other  HPC  capabilities  that  are  required  for  balanced  HPC  systems,  such  as 
central  memory,  secondary  storage,  and  archival  storage.  The  program’s  success  in 
expanding  and  educating  the  DoD  HPC  user  base  will  be  measured  ditatively  based  on 
Hrc  use  in  Defense  applications,  while  quantitative  measures  will  .sed  on  the  increase 
in  the  number  of  new  HPC  applications  and  users.  Success  will  al.s'  .  measured 
qualitatively  by  user  satisfaction  with  the  training,  ease  of  access,  and  ease  of  use  of  new  or 
scalable  systems.  The  metrics  associated  with  the  outreach  program  to  engage  and  leverage 
the  national  HPC  infrastructure  will  include  the  number  of  joint  projects  and  collaborations 
with  academe  and  industry,  cooperative  agreements  that  have  HPC  as  a  key  factor,  and 
evidence  of  transferred  expertise  and  technologies  to  and  from  industry. 


2.5  Outcomes 

Deployment  of  modernized  HPC  provides  an  environment  in  which  the  DoD  S&T 
community  makes  great  strides  using  HPC  to  solve  mission  problems.  Additional  HPC 
resources  allow  researchers  to  tackle  problems  which  are  currently  intractable.  Improved 
HPC  capability  shortens  design  cycles  by  reducing  the  need  to  rely  on  hand-crafted 
prototy^s  and  destructive  testing.  Robust,  high-speed  connectivity  encourages  daily 
collaboration  among  separated  and  remote  users.  Massive  on-line  hierarchic^  data  storage 
coupled  with  interactive  access  to  remote  resources  encourages  users  to  ignore  location  and 
start  considering  hardware  and  software  fit  to  a  particular  problem.  Finally,  lessons 
learned  while  executing  the  above  provides  important  design  feedback  for  future 
generations  of  improved  HPC. 

The  broad  DoD  HPC  system  which  is  the  logical  result  of  this  program  will  be 
widely  recognized  as  a  strategic  resource.  With  these  facilities,  the  E>oD  S&T  community 
will  routinely  conduct  world  class  research,  and  the  knowledge  and  insights  gained  will  be 
crucial  to  maintaining  the  technological  edge  the  U.S.  currently  enjoys.  In  short,  the 
environment  created  by  this  program  will  provide  all  DoD  researchers  the  resources  needed 
to  successfully  execute  the  S&T  mission,  and  will  empower  each  to  access  appropriate 
resources  as  needed  without  leaving  his  or  her  desk  or  workstation. 
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3 .  Leveraging  National  HPC  Infrastructure 


3.1  Program  Relationships 

The  overall  Defense  investment  in  HPC  consists  of  three  interrelated  initiatives:  1) 
participation  in  the  Federal  High  Performance  Computing  and  Communications  (HPCC) 
Program,  2)  Defense-specific  HPC  R&D  programs,  and  3)  the  DoD  HPC  Modernization 
Program,  ^ch  is  a  critically  necessary  element  in  the  application  of  new  HPC  hardware 
and  software  technologies  to  Defense  areas  of  science  and  technology  that  are  essential  to 
the  development  of  new  military  systems.  All  three  programs  involve  extensive  ongoing 
research  and  development  activities  in  academe  and  industry,  as  well  as  the  Federal 
laboratories. 

The  Federal  HPCC  Program  is  a  national  commitment  to  HPC  R&D  and  application 
of  HPC  technology  to  support  die  National  Information  Infrastructure.  For  complete 
details  of  Ae  Federal  HPCC  Program  see  High  Performance  Computing  and 
Communications:  Technology  for  the  National  Information  Infrastructure,  a  report  by  Ae 
Committee  on  Information  and  Communications,  National  Science  and  Technology 
Council. 

The  Defense-specific  HPC  R&D  programs  complement  Ae  Federal  HPCC 
Program.  These  technology  development  programs  focus  on  fulfilling  Defense  HPC  needs 
such  as  providing  tailored  hardware  or  software  elements,  embedded  systems,  accelerators 
for  specific  problem  domains,  and  oAer  challenging  Defense-specific  problems.  Plannmg 
for  these  programs  is  coorAnated  Arough  the  Office  of  Ae  Director,  Defense  Research  and 
Engineering  (DDR&E)  and  through  Ae  Joint  Directors  of  Laboratories  Project  Reliance 
infrastrucAre. 

The  HPC  Modernization  Program  is  a  sustained  activity  designed  to  deploy  high 
performance  computing  systems  and  networks  at  selected  Defense  laboratory  locations  to 
provide  complete  HPC  environments  Aat  will  enable  Defense  scientists  and  engmeers  to 
conduct  important  computational  S&T  m  an  effective  and  efficient  fashion.  The  program 
will  leverage  technology,  applications,  computer  systems,  and  software  products 
developed  Arectly  or  mdirectly  from  DoD  investments  in  Ae  Federal  HrcC  Program, 
Defense-specific  HPC  R&D,  and  other  R&D  efforts. 

WiA  Ae  Federal  HPCC  Program  and  Defense  HPC  R&D  programs  proviAng  Ae 
techmques  and  Ae  HPC  Modernization  Program  providing  Ae  tools  and  environments,  Ae 
Defense  technology  base  programs  will  be  significantly  enhanced.  In  turn,  Aese 
technology  base  programs  develop  Ae  Defense-related  areas  of  S&T  Aat  are  critical  to  Ae 
development  of  new  military  systems. 


3.2  The  Federal  HPCC  Program 

The  goal  of  the  Federal  HPCC  Program  is  to  accelerate  Ae  development  of  future 
generations  of  high  performance  computers  and  networks  and  Ae  use  of  these  resources  m 
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the  Federal  Government  and  throughout  the  American  economy.  Scalable  high 
performance  computers,  advanced  high  speed  computer  communications  networks,  and 
advanced  software  are  critical  components  of  a  new  National  Information  Infrastructure 
(Nil).  This  infrastructure  is  essential  to  our  national  competitiveness  and  will  enable  us  to 
strengthen  and  improve  the  civil  infrastructure,  digital  libraries,  education  and  lifelong 
learning,  energy  management,  the  environment,  health  care,  manufacturing  processes  and 
products,  national  security,  and  public  access  to  government  information. 

Specific  program  goals  include: 

•  Extend  U.S.  technological  leadership  in  high  performance  computing  and 
computer  communications. 

•  Provide  wide  dissemination  and  application  of  the  technologies  to  speed  the 
pace  of  innovation  and  to  improve  the  national  economic  competitiveness, 
national  security,  education,  health  care,  and  the  global  environment. 

•  Provide  a  key  part  of  the  foundation  for  the  National  Information  Infrastructure 
and  demonstrate  selected  Nil  applications. 

The  goals  will  be  realized  through  five  equally  important,  integrated  components 
representing  key  areas  of  high  performance  computing  and  communications: 

•  High  Performance  Computing  Systems  (HPCS)  to  extend  U.S  technological 
leadership  in  high  performance  computing  through  the  development  of  scalable 
computing  systems,  with  associated  software,  capable  of  sustaining  at  least 
one  trillion  operations  per  second  (teraops)  performance. 

•  National  Research  and  Educational  Network  (NREN)  to  extend  U.S. 
technological  leadership  in  computer  communications  by  a  program  of  research 
and  development  that  advances  the  leading  edge  of  networking  technology  and 
services. 

•  Advanced  Software  Technology  and  Algorithms  (ASTA)  to  demonstrate 
prototype  solutions  to  Grand  Challenge  problems  through  the  development  of 
advanced  algorithms  and  software  and  the  use  of  HPCC  resources. 

•  Information  Infrastructure  Technology  and  Applications  (DTA)  to  demonstrate 
prototype  solutions  to  National  Challenge  problems  using  HPCC  enabling 
technologies. 

•  Basic  Research  and  Human  Resources  (BRHR)  to  support  research,  training, 
and  education  in  computer  science,  computer  engineering,  and  computation^ 
science,  and  enhance  the  infrastructure  through  the  addition  of  HPCC 
resources. 

HPCC  program  strategies  include  developing  the  networking  technology  required 
for  deployment  of  nation-wide  gigabit  speed  networks  through  collaborations  with 
indus^;  demonstrating  the  productiveness  of  wide  area  gigabit  networking  to  support  and 
enhance  Grand  Challenge  applications  collaborations;  providing  and  encouraging 
innovation  in  the  use  of  high  performance  computing  systems  and  network  access 
technologies  for  solving  Grand  Challenge  and  other  applications  by  establishing 
collaborations  to  provide  and  improve  emerging  software  and  algorithms;  and  creating  an 
infrastructure,  including  High  Performance  Computing  research  centers,  networks,  and 
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coUaboradons  that  encourage  the  diffusion  and  use  of  high  performance  computing  and 
communications  technologies  in  U.S.  research  and  industri^  applications. 


» 


3.3  Defense  HPC  R&D  Programs 

The  Advanced  Research  Projects  Agency  is  the  lead  DoD  agency  for  advanced 
technology  research  and  development  and  has  the  leadership  responsibility  for  the  Federal 
HFCC  Program  within  DoD.  TTie  ARPA  program  is  developing  the  technologies  needed  to 
enable  a  broad  base  of  applications  and  users  on  new  scalable  computing  systems.  In 
addition,  the  foundations  for  future  generations  of  computing  systems  are  teing  developed. 

In  the  past  year,  the  National  Security  Agency  G'^SA),  has  joined  the  HPCC 
Program.  The  goal  of  NS  As  HPCC  Program  is  to  accelerate  the  development  and 
application  of  the  highest  performance  computing  and  communications  technologies  to  meet 
national  security  requirements. 

The  three  Services  each  have  basic  research  efforts  and  HPC-related  areas  to 
address  fundamental  HPC  research  problems.  These  are  addressed  through  the  Army 
Research  Office,  The  Office  of  Navd  Research,  and  the  Air  Force  Office  of  Scientific 
Research. 

The  Army  Research  Office  (ARO)  has  efforts  in  computational  methods  performed 
by  three  categories  of  investigations:  (i)  single  investigators,  (ii)  block  efforts  and  (iii) 
Centers.  The  single  investigator  studies  vary  from  scalable  software  tools  for  adaptive 
scientific  computation  to  studies  in  elastodynamics.  The  block  efforts  are  funded  primarily 
by  ARPA  and  deal  with  software  development  for  scalable  efforts. 

The  largest  of  the  ARO-sponsored  centers,  the  Army  High  Performance  Computing 
Research  Center  (AHPCRC)  at  the  University  of  Minnesota,  consists  of  three  integrated 
functions,  namely,  (i)  interdisciplin^  research  into  various  aspects  of  HPC  including 
advanced  algorithms,  applications,  implementation  environments,  and  graphics  and 
visualization;  (ii)  evaluation  of  advanced  computing  systems;  and  (iii)  an  infrastructure 
support  program  of  structured  technology  transfer.  A  major  emphasis  is  placed  on  human 
resources  with  significant  emphasis  on  the  involvement  of  Historically  Black  Colleges  and 
Uruversities  and  Minority  Institutions  (HBCUs/MIs). 

The  Office  of  Naval  Research  is  sponsoring  a  five  year  (FY93-97)  multidisciplinary 
basic  research  initiative  in  massive  parallelism  with  the  go^  of  concurrently  accelerating 
computational  research  capabilities  in  Navy-critical  application  domains  while  using  the 
chai^teristics  of  these  applications  to  drive  research  in  advanced  system  software  support, 
programming  environments,  and  numerical  methods.  The  initiative  focuses  on  three 
specific  application  domains  chosen  for  their  mission  criticality  and  the  spectrum  of 
challenges  they  offer  to  scalable  parallel  computing:  large-scale  ocean  circulation 
simulation,  atomic  and  molecular  interaction  calculations  for  advanced  materials,  and  solid- 
phase  propellant  combustion.  Many  of  these  applications  exhibit  irregular,  dynamic,  or 
adaptive  tehavior  inadequately  supported  by  existing  parallel  programming  methodologies 
resdting  in  codes  that  lack  efficiency,  scalability,  or  portability.  Expected  results  include 
efficient,  scalable,  portable  codes  representative  of  the  three  target  application  domains; 
prototype  compilers,  programming  tools,  and  system  software;  and  innovative  parallel 
numerical  algorithms.  Transition  plans  include  widespread  distribution  of  prototype 
software  and  algorithms  and  possible  installation  of  prototype  software  at  DoD  HPC  Major 
Shared  Resource  Centers. 
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Also,  the  Office  of  Navu  Research  manages  a  development  program  to  explore 
HPC  applications  enabling  solution  of  real  Navy  problems.  For  example,  completed  tasks 
have  investigated  applications  to  (1)  increase  range/depth/azimuth  resolution  from  passive 
sonar  arrays,  (2)  perform  2D  and  3D  visualization  of  geographic  data  on  a  massively 
parallel  SI\^  architecture,  and  (3)  accomplish  the  pai^elization  of  a  C3I  mission-planning 
tool  onto  a  heterogeneous  computer  array.  Additional  work  is  ongoing  to  investigate  a 
number  of  Navy  HPC  applications. 

Many  of  the  future  Air  Force  requirements  for  HPC  require  not  only  vast  amounts 
of  computational  power,  but  they  also  require  the  development  of  vastly  improved 
numerical  methods  to  ensure  that  the  application  of  this  computational  power  achieves 
accurate  answers  to  pressing  problems.  The  Air  Force  program,  primarily  through  the  Air 
Force  Office  of  Scientific  Research  (AFOSR)  support  in  its  exter^  program  to  academia, 
concentrates  on  the  development  of  these  novel  methods.  AFOSR  concentrates  on 
supporting  innovative  methods  and  algorithms  which  enable  the  improved  modeling, 
simulation,  understanding,  and  control  of  complex  physical  phenomena. 

The  progress  from  AFOSR  research  efforts  is  utilized  to  impact  critical  applications 
areas,  such  as  computational  plasma  physics,  computational  electromagnetics, 
computational  fluid  dynamics,  and  computational  chemistry.  In  each  of  these  application 
areas,  the  Air  Force,  through  the  AFOSR  basic  research  program  and  other  tectoology 
base  support,  manages  an  integrated  extramural  and  in-house  laboratory  program  to  exploit 
the  potential  of  high  performance  computing. 


3,4  Other  Government  Agency  Programs 

Many  other  government  agencies,  such  as  DOE,  NASA,  and  NSF,  have  extensive 
HPCC  programs.  DoD  laboratories  interact  directly  with  these  agencies  and,  in  addition, 
the  HPC  Modernization  Program  plans  to  interact  more  directly  with  these  agencies  within 
its  various  program  components.  As  the  Modernization  Program  strives  to  provide  a 
complete  HPC  environment  to  the  users  in  the  DoD  S&T  community,  it  recognizes  that 
expertise  in  these  various  agencies  can  make  vital  contributions  to  that  environment. 

Indeed,  collaborative  interactions  currently  exist  among  the  DOE  National  Laboratories,  the 
NSF  HPC  Centers,  and  DoD  laboratories,  including  joint  efforts  in  applications  software 
development.  Several  DoD  laboratories  are  members  of  consortia  involving  Non-DoD 
laboratories.  The  Modernization  Program  will  leverage  that  expertise  through  a  variety  of 
cooperative  HPC  programs. 
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4 .  Requirements  Analysis 


The  DoD  HPC  Modernization  Program  is  designed  to  meet  user  requirements.  In 
April  1994,  a  comprehensive  Requirements  Analysis  was  completed  by  the  HPC 
Modernization  Office,  resulting  in  a  requirements  database  that  is  being  maintained  to 
provide  the  primary  basis  for  future  program  planning. 


4.1  Methodology 

This  Requirements  Analysis  used  information  gathered  via  a  survey  and  interview 
process  with  the  functional  user  community.  It  was  accomplished  by  a  tri-Service  team  of 
eight  scientists  and  engineers  associated  with  HPC,  with  technical  support  from  the 
Institute  for  Defense  Analyses. 

Table  1  lists  organizations  and  locations  that  were  determined  to  have  S&T  HPC 
requirements.  The  total  numbers  of  HPC  S&T  users  within  the  Services  and  participating 
Agencies  are  given  in  Table  2.  Other  agencies,  such  as  the  Defense  Mapping  Agency,  have 
expressed  interest  in  stating  future  requirements  and  will  be  included  as  requirement  are 
periodically  updated. 

The  analysis  determined  functional  S&T  requirements  and  all  aspects  of  HPC 
requirements  that  resulted  from  these  functional  S&T  requirements,  including  hardware, 
software,  networking,  and  training.  A  major  feature  of  the  analysis  was  the  exploration  of 
how  computational  requirements  flow  from  the  underlying  S&T  program  goals,  and,  in 
particular,  to  determine  possible  major  accomplishments  resulting  from  the  exploitation  of 
HPC  capabilities.  An  important  aspect  of  this  activity  is  the  determination  of  the 
impediments  to  routine  use  of  new  scalable  HPC  technologies  by  the  DoD  S&T  program 
and  methods  for  removing  these  impediments. 

The  DoD  computational  S&T  program  was  categorized  into  nine  technology  areas. 
These  areas  are  outlined,  along  with  definitions  of  their  scopes,  in  Table  3.  The  first  six 
areas  represent  collaboration  groups  between  the  DoD  computational  S&T  community  and 
the  National  Consortium  for  High  Performance  Computing  (NCHPC),  designed  to 
accelerate  the  productive  use  of  new  scalable  HPC  technologies  by  the  DoD  S&T  program. 
The  additional  three  areas  were  identified  in  the  Requirements  An^ysis  as  distinct  major 
DoD  computational  technology  areas. 

The  quantitative  and  qualitative  information  was  captured  in  a  data  base  for  display 
and  analysis  and  to  enable  the  program  to  assess  the  evolving  requirements.  Overall  HPC 
performance  requirements  were  aggregated  in  units  of  gigaflops-years  (GF-yrs),  i.e.,  one 
GF-yr  is  the  computational  capacity  represented  by  a  one  GF  processor  computing  for  one 
year.  In  cases  where  user  requirements  were  specified  in  terms  of  specific  HPC  systems, 
these  requirements  were  converted  to  GF-yrs  using  vendor-provided  theoretical  p^ 
poformance  for  each  of  those  systems.  Recognizing  that  actual  efficiencies  vary  widely 
from  system  to  system  and  from  application  to  application,  use  of  theoretical  peak 
performances  to  make  the  conversion  to  GF-yrs  is  considered  to  be  more  consistent  and 
less  arbitrary  than  development  of  subjective  efficiencies  for  typical  DoD  applications. 
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Table  1.  DoD  S&T  Sites 


Organiiatloii 


Anny  Research 
Laboratory 


ank-Automotive 
Research,  Develop- 
ent.  and  Engineering 
Center 


Anny  Corps  of 
Engineers 


Missile  Command 


Locations 


Aberdeen  Proving  Grounds 
(APG),  MD 
Adelpbi,  MD 
Fort  Monmouth,  NJ 
Watertown,  MA 
White  Sands,  NM 


Warren,  MI 


Champaign,  IL 
Ft.  Bel  voir,  VA 
Hanover,  NH 
Vicksburg,  MS 


Huntsville,  AL 


APG,MD 
Frederick,  MD 
Walter  Reed,  MD 


Army  Research  Office  I  Research  Triangle  Park,  NC 


Aviation  and  Troop  Moffett  Field,  CA 

Command  St.  Louis,  MO 


Betbesda,  MD 


APG,  MD 


Armament  Research  Dover,  NJ 

Development  and  Watervliet,  NY 

Engineering  Command 


Edgewood  Research,  APG,  MD 
Development,  and 
Eneineerine  Center 


Organization 

Locations 

Naval  Surface  Warfare 
Center 

Carderock,  MD 

Dahlftren,  VA 

Naval  Air  Warfare 

Center 

China  Lake,  CA 

Patuxent  River,  MD 

Warminster,  PA 

Naval  Command, 

Control  and  Ocean 
Surveillance  Center 

San  Diego,  CA 

Naval  Research 
Laboratory 

Monterey.  CA 

Stennis  Space  Center,  MS 
Washington,  D.C. 

Naval  Undersea  Warfare 

Center 

Newport,  RI 

Office  of  Naval  Research 

Arlington,  VA 

Air  Force  Office  of 

Scientific  Research 

Bolling  AFB,  DC 

USAF  Academy,  CO 

Wright  Laboratory 

Eglin  AFB.  FL 

Tyndall  AFB,  FL 

WPAFB.OH 

Rome  Laboratory 

Griffiss  AFB,  NY 

Hanscom  AFB,  MA 

Edwards  AFB,  CA 

Phillips  Laboratory 

Hanscom  AFB,  MA 

Kirtland  AFB.  NM 

Armstrong 

Laboratory 

Brooks  AFB,  TX 

Tyndall  AFB,  FL 

iDefense  Nuclear  Agency 

Alexandria,  VA  I 

Table  2.  Total  Number  of  HPC  S&T  Users 


Organization 

User  Groups 

Users 

Army 

60 

1,595 

Navy 

93 

757 

AirFwce 

76 

584 

DNA 

4 

330 

Total  DoD 

233 

3,266 
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Tabl0  3.  Computational  Technology  Areas 


Area 

Description 

1 .  Computational  Structural  Mechanics 

•  High  resolution  multi-dimensional  modeling  of  materials 
and  structures  subjected  to  a  broad  range  of  impulsive 
loading  ranging  from  weak  to  intense. 

•  DoD  application  areas  include  conventional  underwater 
explosion  and  ship  response,  structural  acoustics,  coupled 
field  problems,  space  debris,  propulsion  systems, 
structural  analysis,  total  weapon  simulation, 
lethality/survivability  of  weapon  systems  (aircraft,  ships, 
submarines,  tanks),  theater  missile  defense  lethality 
analysis,  and  optimization  techniques. 

2.  Computational  Ruid  Dynamics  I  •  Detailed,  three-dimensional,  high  resolution  Navier-Stokes 


and  large  eddy  simulation  (LES)  modeling  of  fluid  flow  for 
transport  and  tactical  aircraft  at  transonic  through  high 
supersonic  velocities,  projectiles  and  missiles  at  subsonic 
through  hypersonic  velocities,  and  simulated  nuclear  and 
conventionial  weapon  blast  wave  impact  on  tactical 
vehicles  and  structures. 

•  Transient  detailed  reactive  flows  in  two  and  three 
dimensions  for  engine  and  combustor  design,  chemical- 
acoustic  instability  and  noise  control,  gun  tube  propellant 
modeling  which  includes  multiphase  flow,  combustion  of 
solid  and  liquid  propellants,  and  detonation  and  explosion 
dynamics  for  safety,  survivability,  and  lethality. 

•  Detailed,  three-dimensional  transient  hydrodynamic  and 
airwake  flow  about  ships,  submarines,  and  torpedoes 
including  the  multipba^  flows  associated  with  cavitation, 
wavebreaking,  and  underwater  explosions. 

•  Detailed,  medium  and  high  resolution  two-  and  three- 
dimensional  incompressible  flow  modeling  for 
investigating  navigable  waterway/harbor  design, 
vessel/fluid  and  structure/fluid  interactions,  ship-to-sbote 
logistics,  hurricane  protection,  and  hydrologic  systems 
impacts  on  battlefield  mobility. 

•  Area  includes  atmospheric  modeling,  contaminant 
transport  modeling  for  battlefield  and  environmental 
applications,  and  modeling  of  the  cardiovascular  system  in 
humans,  including  blood  flow  through  the  heart 


Table  3.  Computational  Technology  Areas  (Continued) 


Area 

Description 

3.  Computational  Chemistry  and 

Maleaials  Science 

•  Prediction  using  advanced  computational  chemical  and 
molecular  design  tools  to  obtain  accurate  values  of 
detectable  spectroscopic  properties,  evaluate  mechanical 
properties  of  proposed  advaiiced  materials,  and  evaluate  the 
cat^ytic  performance  of  zeolites  and  other  catalytic 
materials  for  destruction  of  hazardous  materials,  such  as 
halons. 

•  Modeling  of  molecular  and  electronic  structures,  reaction 
pathways  and  dynamics,  and  intramolecular  and 
intermolecular  interactions  of  complex  chemical  systems. 

•  Global  optimization  of  extended  molecular  systems. 

4.  Computational  Electromagnetics  and 
Acoustics 

•  High  resolution  multi-dimensional  solutions  of  Maxwell's 
equations  to  define  the  magnetic  fields  about  shipboard 
antenna  arrays,  and  the  electromagnetic  (EM)  signanire«  of 
tactical  ground  and  air  vehicles,  electromagnetic 
performance  and  design  factors  for  EM  gun  technology, 
and  electromagnetic  signatures  of  buried  munitions. 

•  High  resolution  multi-dimensional  solutions  of  the 
acoustic  wave  equations  in  solids,  liquids,  and  gases  to 
model  the  acoustic  fields  of  bodies  of  water  for  underwater 
surveillance  and  communications,  the  seismic  fields  of 
soil  for  mine  detection,  and  the  acoustic  shock  waves  of 
explosions  for  anti-personnel  weapons. 

5.  Climaie/Weatber/Ocean  Modeling 

•  The  modeling  of  the  earth's  climate  and  weather  aimed  at 
improved  prediction  capability. 

*  The  modeling  of  various  properties  of  the  ocean  (e.g., 
temperature,  salinity,  currents)  to  improve  the  processing 
gain  for  acoustic  anti-submarine  warfare  (ASW). 

6.  Signal/Image  Processing 

•  The  extraction  of  useful  information  from  sensor  outputs 
(sonar,  radar,  imaging,  signal  intelligence  (SIGINT), 
navigation),  in  real  time. 

•  Usually  such  processors  are  aboard  deployable  military 
systems  and  hence  require  ruggedized  packaging  and 
minimum  size,  weight,  and  power. 

•  Also  the  research,  evaluation,  and  ofr-line  test  data 
processing  for  these  applications. 

7 .  Forces  Modeling  and  Simulation/C^I 

•  The  use  of  conunand,  control,  communications, 
computers,  and  intelligence  (C^I)  resources  to  manage  a 
battle  space. 

•  Faster-than-real-time  large-scale  simulations  of  complex 
military  engagements  to  facilitate  mission  rehearsal, 
mission  planning,  and  post-mission  analysis. 

•  Collaborative  planning/replanning  to  support  real-time 
operation  decision-making. 

Tabis  3.  Computational  Tachnoiogy  Araas  (Concludad) 


Area 

Description 

8.  Environmental  Quality  Modeling  and 
Simulation 

•  High  resolution  three-dimensional  Navier-Stokes  modeling 
of  hydrodynamics  and  contaminant  and  multi-constituent 
fate/transpon  through  the  aquatic  and  terrestrial  ecosystem 
and  wetland  subsystems,  and  their  coupled  hydrogeologic 
pathways,  and  their  interconnections  with  numerous 
biological  species. 

•  Used  for  stewardship  and  conservation  of  natural  and 
cultural  resources,  optimal  design  and  operation  of 
installation  restoration,  and  enhancement  alternatives  and 
development  of  shoit-  and  long-term  strategies  for 
integrated  management  in  support  of  installation 
environmental  quality. 

•  Work  in  the  area  of  noise  evaluation  and  abatement  as  well 
as  water  quality  models. 

9.  Computational  Electronics  and  Nano¬ 
electronics 

•  Simulation  of  electronic  devices  in  circuitry,  including 
fault  simulation. 

•  Monte  Carlo  and  nonlinear  quantum  transport 
characterization  in  nanoelectronic  devices. 

•  Electronic  struchire  calculations  oi  novel  materials  for 
electronics  applications. 

4.2  Summary  of  Requirements  Analysis 

The  DoD  science  and  technology  program  has  a  large  number  of  functional 
requirements  which  can  be  satisfied  by  rapid  increases  in  HPC  capabilities.  These 
functional  S&T  requirements  have  been  identified  both  as  a  set  of  grand  challenge  problems 
spanning  the  nine  computational  science  technology  areas  and  as  the  S&T  goals  of  each 
individual  DoD  laboratory  location.  These  functional  S&T  requirements  result  in  overall 
HPC  performance  requirements  for  FY94,  FY96,  and  FY99  as  summarized  by  Service  in 
Tables  4  and  5. 


Table  4.  Summary  of  Overall  Unclassified  Performance 
Requirements  (GF-Yrs) 


ORGANIZATION 

FY94 

FY96 

FY99 

TOTAL  ARMY 

297 

3,829 

27,119 

TOTAL  NAVY 

138 

1,196 

’5,679 

TOTAL  AIR  FORCE 

180 

2,032 

19,668 

TOTAL  DNA 

1 

1 

2 

TOTAL 

616 

7,058 

62,468 

19 


Table  5.  Summary  of  Overall  Classified  Performance 
Requirements  (GF*Yrs) 


ORGANIZATION 

FY94 

FY96 

FY99 

TOTAL  ARMY 

21 

266 

1.735 

TOTAL  NAVY 

5 

25 

158 

TOTAL  AIR  FORCE 

16 

266 

3,021 

TOTAL  DNA 

0 

0 

0 

TOTAL 

42 

557 

4,914 

Each  Service  has  rapidly  increasing  requirements  for  HPC  capability  in  its  S&T 
program  over  this  five-year  time  period.  As  modeling  and  simulation  activities  play  an 
increasingly  larger  role  in  DoD’s  R&D  program,  driven  by  the  need  to  maintain 
technological  superiority  in  this  downsizing  era,  it  becomes  crucial  that  advances  in  HPC 
capabilities  are  rapidly  made  available  to  the  DoD  S&T  computational  community.  As  the 
tables  indicate,  the  computational  S&T  community  has  significant  requirements  for  and  is 
ready  to  use  teracomputing  capability  and  beyond  within  the  next  five  years  to  advance  the 
S&T  programs  identified  in  this  report  so  that  DoD  can  maintain  its  superiority  in 
warfighting  systems. 

The  Requirements  Analysis  has  confirmed  the  need  for  a  full  range  of  HPC  system 
sizes,  with  users  citing  requirements  from  workstations,  to  clusters,  to  scalable  parallel 
systems,  to  conventional  supercomputers  to  accomplish  their  S&T  computational  mission. 
To  use  the  high-end  systems  to  their  maximum  effectiveness,  many  users  require  a  variety 
of  individual  and  clustered  systems  to  pre-  and  post-process  large  production  computations 
performed  on  the  high-end  systems.  There  is  a  common  need  for  scientific  visualization 
systems  to  allow  scientists  and  engineers  to  make  effective  use  of  computational  results. 
lYoviding  a  range  of  system  sizes  ensures  that  scientific  and  engineering  problems  of 
varying  sizes  can  be  accomplished  on  the  most  appropriate  system,  thus  preserving  high- 
end  system  resources  for  the  largest  problems. 

It  is  widely  accepted  that  no  single  architecture  or  system  type  will  fulfill  all  of  the 
computational  requirements  of  the  DoD  S&T  program.  Systems  that  provide  significant 
growth  capability  are  essential  to  satisfying  the  rapidly  growing  computational  S&T 
requirements  identified  in  the  Requirements  Analysis. 


A  comparison  between  Tables  4  and  5  clearly  shows  that  the  bulk  of  the  HPC 
requirements  of  the  DoD  S&T  program  is  unclassified.  The  classified  requirement  is 
probably  understated,  however,  due  to  the  scarcity  of  available  classified  HPC  resources  at 
this  time.  Many  scientists  and  engineers  tend  to  compute  problems  that  are  unclassified  and 
use  those  results  to  extrapolate,  when  necessary,  to  actual  systems  whose  attributes  are 
classified.  Even  though  overall  performance  requirements  for  classified  S&T  computing 
are  a  relatively  small  fraction  of  the  total  requirement,  those  classified  requirements 
represent  a  key  ability  for  DoD  to  respond  quickly  in  any  time  of  crisis  to  developmental 
and  operational  needs. 

In  addition  to  computational  capabilities,  other  key  hardware  requirements  such  as 
central  memory,  secondary  storage,  and  archival  storage  were  gathered  and  analyzed.  A 
requirement  for  very  large  central  memory,  secondary,  and  archival  storage  was  validated 
for  any  mode  of  operation  of  a  large  HPC  system,  whether  it  be  to  provide  maximum 
throughput  for  a  large  number  of  DoD  S&T  users  representing  a  broad  mix  of  jobs,  or  to 
facilitate  a  limited  number  of  very  large  grand  challenge  DoD  S&T  applications.  Archival 
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storage  requirements  will  quickly  expand  to  petabytes  (10^5  bytes)  as  more  complex 
computational  problems  are  attacked,  with  resultant  increase  in  output  tile  size.  Automated 
storage  capabilities  are  essential  to  ensure  timely  access  to  data  for  the  large  and  growing 
computational  S&T  user  base.  Anticipated  leaps  in  computational  power  will  be  essentially 
wasted  if  these  central  memory,  secondary  storage,  and  archival  storage  capabilities  do  not 
keep  pace. 

Software  requirements  were  identified  as  the  one  key  element  in  ensuring  that  DoD 
computational  scientists  and  engineers  would  be  capable  of  taking  full  advantage  of  the 
vastly  increased  computational  capabilities  of  new  scalable  architectures.  It  is  essential  that 
mature  operating  systems  and  system  tools  be  made  available  as  soon  as  possible  in  a  new 
system's  life  cycle  so  that  its  capabilities  can  be  fully  exploited.  In  addition,  robust 
parallelization  tools  will  help  ensure  compatibility  of  applications  codes  across  different 
computational  platforms.  These  parallelization  tools  include  new  standard  compilers, 
debuggers,  performance  monitors,  and  other  parallelization  aids.  One  promising  method  of 
accelerating  the  transition  of  existing  DoD  applications  codes  to  new  scalable  architectures 
is  teaming  computer  scientists  with  an  expertise  in  parallel  programming  with  DoD 
computational  scientists  who  are  knowledgeable  about  the  computation^  problems  being 
addressed.  In  general,  most  computational  scientists  express  a  strong  desire  not  to  be 
computer  scientists,  but  to  spend  their  time  doing  S&T  in  the  field  of  their  training.  This 
teaming  would  enable  the  development  of  expertise  in  the  exploitation  of  scalable  systems 
within  the  DoD  S&T  program,  and  could  be  facilitated  by  placing  graduate  and  postdoctoral 
students  and  other  temporary  personnel  with  such  expertise  from  academic  computer 
science  departments  into  DoD  laboratories. 

Networking  is  a  critical  ingredient  in  the  overall  ability  to  ensure  accessibility  to 
state-of-the-an  HFC  capabilities  across  the  entire  DoD  computational  S&T  program.  It  is 
clear  thsi  every  DoD  S&T  site  will  not  be  hosting  a  major  HPC  system  and  most  sites  will 
necessarily  access  the  high-end  systems  remotely.  The  data  gathered  in  the  requirements 
survey  indicate  high  performance  and  comprehensive  networking  capabilities  are  required 
to  support  effective,  remote  use  of  HPC  resources.  A  prevalent  and  preferred  mode  of 
remote  processing  is  to  return  all  or  most  output  data  tiles  to  the  local  S&T  site,  enabling, 
for  example,  postprocessing  and  visualization  rendering.  These  output  data  tiles  are 
typically  quite  large  and  create  large  network  throughput  demands  if  timely  transfers  are  to 
be  achieved.  Additionally,  an  eventual  goal  of  real-time,  interactive  visualization  performed 
on  remote  major  resource  center  assets  will  require  gigabit/second  network  bandwidth  end 
to  end.  Postprocessing  output  data  tiles  at  loc^  S&T  sites  further  introduces  a  requirement 
for  large  capacity  archival  storage  resources  at  these  local  sites.  A  distributed  data  storage 
architecture,  enabled  by  an  effective  high  performance  network,  has  the  benefit  of  reducing 
long-term  storage  demands  at  the  Major  Shared  Resource  Centers. 

To  take  advantage  of  projected  Defense  Research  and  Engineering  Network 
(DREN)  capabilities  to  meet  ^ese  large  networking  requirements  between  Major  Shared 
Resource  Centers,  Distributed  Centers,  collaboration  parmers,  and  user  sites,  each  site 
must  establish  and  support  a  local  networking  infrastructure  capable  of  interfacing  with 
DREN  wide  area  services  and  be  appropriately  scaled  to  handle  DREN-supported  data 
rates.  These  local  networking  capabilities  are  the  responsibility  of  each  locd  site  and  failure 
to  support  an  adequate  local  infrastructure  will  impede  its  scientists  and  engineers  in  their 
efforts  to  exploit  DoD  HPC  resources  to  their  full  potential. 

A  final  and  critical  aspect  of  networking  is  a  classified  capability.  Classified  S&T 
computing  represents  a  key  DoD  capability  to  respond  quickly  in  times  of  crisis.  Current 
encryption  technology  severely  lags  network  bandwidth  performance  and  hampers  the 
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ability  of  the  DoD  Modernization  Program  to  provide  remote,  high  performance  classified 
HPC  capability.  Until  this  encryption  performance  deficiency  is  resolved,  there  will 
continue  to  be  critical  classified  HPC  r^uirements  which  can  only  be  met  in  the  near-term 
with  local  classified  resources.  As  part  of  the  DREN  initiative,  developments  in  advanced 
encryption  technology  will  be  closely  monitored  and  earliest  possible  deployment  of 
approved  products  are  planned. 

Training  and  education  requirements  likewise  focus  on  efficient  use  of  new  scalable 
HPC  architectures.  These  requirements  include  specific  HPC  systems  and  efficient  parallel 
programming  techniques  in  general.  Long-term,  full-time  programs  will  be  necessary  to 
provide  training  and  education  in  the  use  and  programming  of  scalable  systems. 
Requirements  dso  exist,  but  to  a  lesser  extent,  for  systems  software  such  as  Unix,  Fortran, 
Ada,  and  C,  and  for  effective  use  of  applications  codes.  For  all  training  and  education,  a 
two-tiered  system  seems  most  effective:  an  introductory  level  for  novices  and  an  advanced 
level  for  more  experienced  users.  Most  users  prefer  training  that  is  provided  in  their  local 
areas,  but  recognize  the  need  for  a  critical  mass  to  ensure  the  cost  effectiveness  of  that 
training. 

The  overall  conclusion  is  that  a  complete  HPC  environment  must  be  provided  to 
take  full  advantage  of  HPC  capabilities  within  the  DoD  S&T  program.  This  environment 
must  include  a  variety  of  HPC  systems  to  provide  the  proper  computational  platforms  for 
which  computational  problems  will  run  most  efficiently.  Raw  computational  capability 
must  be  balanced  by  a  compatible  central  memory,  secondary  storage,  and  archival  storage 
capability.  Systems  software  and  efficient  applications  software  must  be  available  to  allow 
the  DoD  scientist  and  engineer  to  compute  effectively  on  HPC  platforms  and  thus  perform 
the  DoD  S&T  mission.  Special  programs  to  £«;celerate  the  porting  and  development  of 
applications  software  to  fully  utilize  new  scalable  architectures  are  essential  for  the  S&T 
computational  program  to  stay  abreast  of  new  HPC  capabilities.  A  reliable  high-speed 
network  is  the  crucial  link  between  computational  scientists  and  engineers  at  DoD 
laboratories  and  the  high-end  HPC  resources  located  at  DoD  Major  Shared  Resource 
Centers.  Training  and  education  is  essential  to  keep  the  user  base  technically  current. 


5.  Implementation  Strategy 


5.1  Consolidation  of  DoD’s  HPC  Infrastructure 

The  planned  HPC  infrastructure  will  consist  of  both  centralized  HPC  resources 
located  at  MSRCs  and  decentralized  HPC  resources  located  at  Distributed  HPC  Centers. 
As  discussed  earlier,  these  resources  will  be  available  to  S&T  researchers  and  engineers 
throughout  the  DoD  laboratories  and  R&D  centers  via  DREN.  The  roadmap/timetable  for 
advancing  the  capabilities  of  this  infrastructure  is  depicted  in  Figure  3. 

Efforts  conducted  in  FY93  and  FY94  represent  initial  steps  in  advancing  the  HPC 
capabilities  in  the  infrastructure.  Providing  a  large  Cray  C90  system  at  CEWES  in  FY93 
represented  the  first  step  in  establishing  a  MSRC.  Providing  other  laboratories  and  centers 
with  new  scalable  systems  and  upgrad^  to  existing  scalable  systems  in  FY93  was  the  Erst 
step  in  establishing  Distributed  HITC  Centers.  Establishing  the  Interim  DREN  by 
integrating  the  existing  Army  Supercomputer  Network  and  the  Air  Force  Supercomputer 
Network,  and  linking  them  to  Navy  and  DNA  centers,  initiated  the  sharing  of  these  HPC 
resources  across  all  the  Services  and  participating  agencies.  This  action  began  the 
replacement  of  individual  Services’  and  Agency’s  ETC  centers  with  a  consolidated  DoD 
HTC  infrastructure. 

In  FY94  these  actions  are  continued  with  the  initiation  of  an  additional  MSRC  and 
several  new  Distributed  HPC  Centers,  as  well  as  upgrades  to  the  first  MSRC  and  some  of 
the  existing  Distributed  Centers.  The  major  efforts  to  initiate  the  long  term  strategy  will  be 
conducted  in  FY95  and  continued  in  FY96  and  beyond.  These  actions  are  described  in 
more  detail  in  the  following  sections. 


5.2  FY93  Efforts 

Section  1.2  outlined  the  actions  taken  to  provide  additional  HPC  capabilities  to  the 
DoD  S&T  community  as  approved  in  the  FY93  Implementation  Plan  using  the  combined 
funds  provided  by  Congress  in  FY92  and  FY93.  Those  actions  have  made  significant 
enhancements  to  the  DoD  S&T  program  and  have  begun  to  satisfy  the  DoD  S&T 
computational  requirements  as  summarized  in  Chapter  4. 

The  addition  of  the  Cray  C916f512  and  the  conversion  of  the  existing  Y-MP8  at 
CEWES  to  DoD  operations  represents  approximately  a  doubling  of  the  vector  HPC 
c^ability  previously  available  to  the  DoD  S&T  program.  While  its  overall  HPC 
performance  (16  GF-yrs)  is  only  a  small  fraction  of  the  616  GF-yrs  unclassified  HPC 
requirements  for  FY94  stated  in  Table  4,  it  has  allowed  DoD  researchers  to  attack 
significantly  larger  problems  than  they  could  previously.  One  crucial  limitation  has  been 
the  amount  of  centi^  memoiy,  secondary  storage,  and  archival  storage  available  on  that 
system. 
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Figure  3.  DoD  HPC  Modernization 
Pian  Roadmap 


The  set  of  scalable  parallel  systems  acquired  or  enhanced  as  part  of  the  FY93 
Implementation  Plan  represents,  in  the  aggregate,  over  200  GF-yrs  of  overall  HPC 
performance  available  to  DoD  S&T  users.  Tim  capability  is  a  significant  fraction  of  the 
total  HPC  requirements  for  FY94  (Tables  4  and  5).  In  addition,  the  Intel  Paragon  XP/S-25 
operated  at  NCCOSC  in  a  classified  mode  provides  a  substantial  part  of  the  42  GF-yrs  of 
classified  S&T  requirements,  but  not  all  classified  users  are  capable  of  taking  advantage  of 
that  particular  architecture.  Many  of  the  scalable  parallel  systems  implemented  in  FY93 
lack  adequate  system  peripherals  and  central  memory  to  allow  efficient  use  by  multiple 
users. 

Interim  DREN  has  proven  to  be  highly  successful  in  allowing  users  at  DoD  user 
sites  to  access  the  HPC  systems  implemented  in  FY93.  Key  limitations  include  insufficient 
bandwidth  to  move  large  files  created  by  some  user  applications  and  the  slower 
transmission  speeds  available  to  classified  users  who  must  depend  on  end-to-end 
encryption  devices. 

The  MOA  with  NCHPC  has  recently  begun  to  bear  fruit  for  DoD  S&T  users.  The 
NCHPC  collaboration  groups  have  held  initial  meetings  and  begun  to  formulate  plans  for 
scalable  software  development  efforts  in  each  area.  One  indirect  benefit  of  the  NCHPC 
collaboration  groups  has  been  the  incentive  for  internal  DoD  researchers  within  each 
technology  group  to  meet  and  collaborate  in  ways  that  have  not  occurred  previously. 
Several  DoD  research  projects  have  made  use  of  NCHPC  computational  resources  that 
have  allowed  scaling  problems  to  larger  sizes  than  would  have  been  possible  on  DoD  HPC 
systems  alone.  To  ^te,  this  added  capability  represents  only  a  tiny  fraction  of  the  total 
DoD  HPC  S&T  requirements,  but  further  exploitation  of  NCHPC  assets,  based  on  the  FY 
92/93  investment,  will  allow  unique  opportunities  for  specialized  DoD  applications. 

5.3  FY94  Efforts 

An  Implementation  Plan  has  been  developed  in  close  coordination  with  the  Services 
and  S&T  user  community  to  apply  available  FY94  funds  to  immediate  upgrades  and 
expansions,  to  alleviate  current  saturation,  and  to  make  existing  systems  more  effective. 
The  DoD  S&T  community  has  significant  requirements  to  solve  research  problems  in  the 
memory  and  compute  intensive  domain  for  which  high  performance  computing  is  required. 
Table  6  indicates  the  HPC  Center  deployments  that  represent  critical  S&T  requirements  that 
can  be  addressed  immediately  through  FY94  acquisitions.  Deployments  to  each  of  these 
centers  were  judged  using  the  criteria  of  S&T  requirements,  economic  soundness 
management  commitment,  and  information  infrastructure.  Execution  of  the  plan  will 
involve  the  use  of  both  new  and  previously  competed  contracts  and  be  in  full  compliance 
with  Congressional  direction  for  FY94.  The  Implementation  Plan  contains  a  system 
summary,  rationale,  acquisition  plan,  and  funding  allocation  for  each  of  the  planned  FY94 
acquisitions.  A  similar  Implementation  Plan  will  be  issued  early  each  FY  to  provide  like 
details  for  that  FY’s  program  execution. 


Tabis  6.  Summary  of  Planned  FY94  HPC  Systems 


SITE 

SYSTEM 

FY94  ACTIONS 

CEWES 

C90 

Upgrades  to  central  memory,  secondary  storage,  and  archival 
storace 

NAVO 

C90 

New  Cray  C916/1024 

Anny 

Res^rcfaLab 

TBD 

New  scalable  HPC  system 

ASC 

Unstade  to  distributed  memory  and  secondary  storage 

AFDTC 

CravT3D 

New  Cray  T3D  scalable  system 

MHPCC 

IBM  SP2 

Upgrade  of  performance  capability  from  60  to  100  GF  fw 
unclassified  system;  upgrades  to  distributed  memory  and 
secondary  storage 

NCCOSC 

Paragon 

Upgrade  of  distributed  memory  and  secondary  storage; 
addition  of  file  archiving  system 

SPP-1 

Upgrade  of  distributed  memory  and  other  peripherals; 
additional  processors 

Naval 

CM-5 

Upgrade  to  distributed  memory  and  secondary  storage; 

Research 

additional  processors 

TBD 

New  scalable  HPC  system 

Rome  Lab 

TBD 

New  scalable  HPC  system 

In  addition  to  addressing  some  specific  local  requirements  detailed  in  the 
Implementation  Plan,  these  FY94  actions  address  a  broader  base  of  requirements  as 
summarized  in  Chapter  4  and  as  determined  by  initial  experiences  with  systems 
implemented  in  FY93.  Several  existing  systems,  including  the  C90  at  CEWES,  the  IBM 
SP2  at  the  MHPCC,  the  Paragons  at  NCCOSC  and  ASC,  and  the  CM-5  at  NRL,  are 
having  either  central  memory  or  peripheral  storage  upgraded  to  make  those  systems  more 
efficient  and  effective  for  current  and  projected  user  bases.  In  each  case,  user  requirements 
and  experiences  dictate  that  these  enhancements  are  essential  to  continued  efficient 
operations.  In  each  of  these  cases,  the  initial  FY92/93  investment  provided  the  basic 
system,  but  it  was  realized  at  that  time  that  FY94  upgrades  would  have  to  be  made  to 
complete  a  balanced  system. 

New  capacity  and  capability  is  also  a  component  of  the  FY94  actions.  The  addition 
of  a  new  Cray  C916/1024  and  the  conversion  of  the  existing  Y-MP/8  to  classified  DoD  use 
at  the  Naval  Oceanographic  Office  (NAVO)  will  further  enhance  DoD  vector  HPC 
capability,  alleviate  some  of  the  oversubscription  of  existing  systems  at  CEWES,  allow 
more  extensive  “grand  challenge”  computations  on  these  systems,  and  provide  the  first 
DoD  vector  HPC  classified  capability.  All  of  these  improvements  in  the  ability  to  satisfy 
current  user  requirements  are  urgently  needed. 

New  scalable  parallel  systems  at  AFDTC,  ARL,  NRL,  and  Rome  Laboratory,  with 
computational  enhancements  planned  to  existing  scalable  parallel  systems  at  the  MHPCC 
and  NCCOSC,  will  add  over  100  GF-yrs  to  the  overall  HPC  performance  available  to  DoD 
S&T  users.  When  the  enhancements  are  combined  with  the  capabilities  provided  in  FY93 
and  the  new  vector  HPC  capabilities  at  NAVO  in  FY94,  the  total  performance  capability 
available  to  DoD  HPC  researchers  will  be  approximately  400  GF-yrs.  This  total  represents 
60  per  cent  of  the  total  FY94  S&T  requirements  given  in  Tables  4  and  5. 
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5.4 


Planned  FY95  Efforts 


To  fulfill  the  goal  of  providing  HPC  capacity  and  capability,  and  to  address  the 
software  issues  in  training  and  education  required  to  effectively  use  the  HPC  resources, 
the  program  will  continue  its  plans  to  establish  DoD  Major  Shared  Resource  Centers 
(MSRCs)  located  within  the  Services  and  participating  Agencies.  The  MSRCs  will  provide 
complete  HPC  environments  and  include  various  types  of  computing  systems,  scientific 
visu^ization  capabilities,  extensive  peripheral  and  archival  storage,  and  expertise  in  use  of 
these  systems.  The  MSRCs  will  support  the  wide  variety  of  research  and  development 
problems  arising  from  the  science  and  technology  programs  supporting  DoD's  weapons 
development  and  warfighting  support  systems.  The  MSRCs  will  provide  the  computer 
and  computational  sciences  expertise  to  allow  all  of  the  DoD  laboratories  to  advance  their 
capability  in  science  and  technology.  The  t3^s  of  computer  systems  in  the  MSRCs  will  be 
determined  by  user  requirements  and  may  differ  from  one  MSRC  to  another. 

Actions  to  establish  MSRCs  are: 


•  The  program  has  selected  four  sites  to  become  MSRCs.  Criteria  for  selection 
were  based  on  R&D  synergism,  infrastructure,  affordability,  management 
commitment,  and  experience  in  serving  remote  users.  The  selected  sites  for 
MSRCs  are: 

Army  Corps  of  Engineers  Waterways  Experiment  Station,  Vicksburg,  MS 
Naval  Oceanographic  Office,  Stennis  Space  Center,  MS 
-  Army  Research  Laboratory,  Aberdeen  Proving  Ground,  MD 

Aeronautical  Systems  Center,  Wright-Patterson  Air  Force  Base,  OH 


•  In  FY95  the  program  office  is  planning  a  full  and  open  competition  for 
acquisition  of  HPC  capabilities,  based  on  requirements  of  the  users,  to 
populate  the  MSRCs.  The  program  anticipates  multi-awards  to  teams 
including  academe  and  industry,  led  by  system  integrators. 

In  addition  to  MSRCs,  the  program  will  continue  to  deploy  systems  to  Distributed 
HPC  Centers  to  fulfill  special  R&D  requirements  where  the  HPC  systems  are  critical  to 
success  of  S&T  efforts  and  where  special  collections  of  HPC  expertise  can  be  leveraged  for 
benefit  of  the  entire  program.  Multiple  systems  will  be  acquired  using  several  competitive 
acquisition  vehicles. 

Figure  4  depicts  the  DoD  HPC  laboratory  modernization  of  both  MSRCs  and 
Distributed  Centers.  All  functions  are  performed  by  the  DoD  HPC  MSRCs.  The  shaded 
functions  in  the  figure  represent  those  Inunctions  expected  of  the  Distributed  HPC  Centers. 

In  FY95  the  program  will  initiate  a  full  and  open  competition  for  acquisition  of 
network  services  to  provide  OC3-level  service  (155  Mbps)  to  selected  sites  and  minimum 
T3  (45  Mbps)  service  to  all  other  DoD  sites  participating  in  the  program.  Award  of  a 
contract  for  DREN  network  services  is  planned  for  the  third  quarter  of  FY95.  These 
services  will  replace  the  Interim  DREN,  which  as  described  earlier,  is  comprised  of 
dedicated  leased  circuits.  Anticipated  benefits  include  higher  performance  and  improved 
cost  effectiveness. 
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Distributed  HPC  Centers 
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Figure  4.  DoD  HPC  Laboratory  Modernization 


The  program  will  leverage  existing  resources  and  university  relationships  for  HPC 
capability,  training,  applications  and  algorithm  development,  and  software  tools.  It  will 
continue  to  coordinate  with  NCHPC  to  take  advantage  of  available  training  through  the 
NCHPC  members,  continue  software  collaborations,  and  make  use  of  the  NCHrc 
software  repository  and  NCHPC  scalable  hardware.  Finally,  the  program  is  also  planning 
to  implement  an  internal  DoD  scalable  applications  software  initiative  to  enable  DoD 
researchers  to  begin  to  overcome  the  software  impediments  to  effective  use  of  scalable 
HPC  on  DoD  mission  problems. 

The  summaries  of  classified  and  unclassified  requirements  in  Tables  4  and  5 
indicate  an  order  of  magnitude  growth  from  FY94  to  FY96.  The  planned  FY95  activities, 
particularly  those  estabUshing  and  equipping  the  MSRCs,  must  put  in  place  a  large  share  of 
HPC  systems  and  environments  targeted  to  meet  those  FY96  needs.  The  total  requirement 
of  over  seven  TF-yrs  will  be  difficult  to  meet  with  planned  FY95  activities  alone.  Current 
projections  indicate  that  the  initial  acquisitions  for  ^e  MSRCs  in  FY9S  could  add  several 
hundred  additional  GF-yrs  to  the  available  DoD  capability,  resulting  in  a  total  capability 
approaching  one  TF-yr. 
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5.5  Long  Term  Implementation  Strategy  (FY96-FY99) 

•  Continue  to  evolve  and  modernize  MSRCs. 


•  Continue  to  acquire  special  systems  for  Distributed  HPC  Centers  with  critical 
requirements. 

•  Acquire  adequate  data  storage  capabilities  at  appropriate  distributed  centers. 

•  Evolve  the  network  to  gigabit  per  second  (and  beyond)  performance  levels  to  be 
consistent  with  the  performance  levels  of  the  HPC  systems,  and  as  user 
requirements  dictate. 

•  Develop  mechanisms  for  creation  of  interdisciplinary  teaming  which  leverages 
academe  and  industry  expertise  in  HPC. 

•  Promote  software  tooling  and  scalable  application  and  algorithm  development 
initiatives. 

If  it  is  difficult  to  meet  the  FY96  S&T  computational  requirements,  it  is  even  more 
difficult  to  satisfy  the  projected  FY99  computational  requirements  of  70  TF-yrs.  These 
planned  out-year  actions  are  likely  to  result  in  a  total  DoD  S&T  performance  capability  of 
20-30  TF-yrs.  This  level  of  capability  would  represent  slightly  less  than  half  of  the 
projected  computational  requirements. 


5.6  Funding  Needs 


Table  7.  Funding  Profile  ($M) 


FY94 

FY95 

FY96 

FY97 

FY98 

FY99 

MSRC  Systems^ 

73.2 

80.0 

80.0 

80.0 

95.0 

100.0 

Distributed  Center  Systems^ 

49.1 

40.0 

40.0 

40.0 

55.0 

Software,  Appl  Support,  Education 
and  Training  Initiatives 

0.82 

20.0 

25.0 

30.0 

40.0 

50.0 

Networking 

10.8 

20.0 

30.0 

40.0 

45.0 

50.0 

Program  Management  and 
Administration 

3.0 

mgin 

3.2 

■1 

3.5 

m 

Sustainment  of  Operations 

6.9 

44.0 

49.8 

55.0 

TOTAL 

143.8^ 

183.0 

211.1 

237.4 

288.3 

313.8 

^  Hiese  systems  recognize  that  there  are  specialized  requirements  both  mission  and  contractual  (such 
as  DNA’s  contractual  requirements)  which  must  be  met  by  the  HPC  MP  and  provisions  will  be 
made  to  meet  these  needs 

^  Includes  $0.8M  COTS  software 

3  Available  funds  after  undistributed  cuts  ($146.1M  apprq>nated) 
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6.  Acronyms  and  Definitions 


AFB 

Air  Force  Base 

AFDTC 

Air  Force  Development  Test  Center 

AHPCRC 

Army  High  Performance  Computing  Research  Center 

algorithm 

procedure  designed  to  solve  a  problem 

AMOS 

Air  Force  Maui  Optical  Station 

APG 

Aberdeen  Proving  Ground 

ARL 

Army  Research  Laboratory 

ARPA 

Advanced  Research  Projects  Agency 

ASC 

Aeronautical  Systems  Center 

ASD 

Assistant  Secretary  of  Defense 

ASW 

anti-submarine  warfare 

CAA 

Concepts  Analysis  Agency 

C3I 

Command,  Control,  Communications,  and  Intelligence 

C4I 

Command,  Control,  Communications,  Computers  and 
Intelligence 

CEWES 

Army  Corps  of  Engineers  Waterways  Experiment  Station 

Computational  Science 
Engineering 

The  systematic  application  of  computing  systems  and 
computational  solution  techniques  to  mathematical  models 
formulated  to  describe  and  simulate  phenomena  of  scientific 
and  engineering  interest. 

CPU 

central  processing  unit 

DDR&E 

Director  of  Defense  Research  and  Engineering 

DNA 

Defense  Nuclear  Agency 

DoD 

Department  of  Defense 

DOE 

Department  of  Energy 

DREN 

Defense  Research  and  Engineering  Network 

FFRDC 

Federally  Funded  Research  and  Development  Centers 
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FY 

GF 

gflop 

Grand  Challenge 

HBCU 

Heterogeneous  System 
HPC 


HPCC 

HPCM 

HPCWG 

IDREN 

Internet 

LES 

MHPCC 

MI 

MOA 

MSRC 

NAVO 

NASA 

NCCOSC 


fiscal  year 

gigaflops 

gigaflops 

A  fundamental  problem  in  science  and  engineering  with 
broad  economic  and  scientific  impact,  whose  solution  can  be 
advanced  by  applying  HPC  techniques  and  resources. 

Historically  Black  Colleges  and  Universities 

A  distributed  system  that  contains  more  than  one  kind  of 
computer. 

High  Performance  Computing 

Covers  the  full  range  of  advanced  computing  systems 
including  workstations,  networks  of  workstations  with 
servers,  scalable  parallel  systems,  vector  parallel  systems, 
and  more  specialized  systems.  Scalable  input/output 
interfaces,  mass  storage  systems,  and  archival  storage  are 
components  of  these  systems.  Included  also  are  system 
software  and  software  development  environments  that  enable 
users  to  view  their  workstations  and  the  rest  of  their 
computing  environments  as  a  unified  system. 

High  Performance  Computing  and  Communications 

High  Performance  Computing  Modernization 

High  Performance  Computing  Working  Group 

Interim  Defense  Research  and  Engineering  Network 

The  global  collection  of  interconnected  multi  protocol 
computer  networks. 

large  eddy  simulation 

Maui  High  Performance  Computing  Center 

Minority  Institutions 

Memorandum  of  Agreement 

Major  Shared  Resource  Center 

Naval  Oceanographic  Office 

National  Aeronautics  and  Space  Administration 

Naval  Command,  Control  and  Ocean  Surveillance  Center 
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NCHPC 

NRL 

NSF 

NSWC 

NUWC 

OGA 

ONR 

R&D 

RDT&E 

S&E 

SRC 

S&T 

TBD 

TF 


National  Consortium  for  High  Performance  Computing 

Naval  Research  Laboratory 

National  Science  Foundation 

Naval  SurfaceWarfare  Center 

Naval  Undersea  Warfare  Center 

other  government  agency 

Office  of  Naval  Research 

Research  and  Development 

Research.  Development,  Test,  and  Evaluation 

scientists  and  engineers 

Shared  Resource  Center 

science  and  technology 

to  be  determined 

teraflops 

Wright-Patterson  Air  Force  Base 


WPAFB 


